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DETAILED ACTION 



1. 



Claims 1-40 are pending and have been examined. 



2. 



Claims 1-40 are rejected. 



Drawings 



3. The drawings are objected to under 37 CFR 1 .83(a). The drawings must show 
every feature of the invention specified in the claims. Therefore, the query system 
recited in claims 1-38 and the method steps recited in claims 39-40 must be shown or 
the feature(s) canceled from the claim(s). No new matter should be entered. 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. The figure or figure 
number of an amended drawing should not be labeled as "amended." If a drawing figure 
is to be canceled, the appropriate figure must be removed from the replacement sheet, 
and where necessary, the remaining figures must be renumbered and appropriate 
changes made to the brief description of the several views of the drawings for 
consistency. Additional replacement sheets may be necessary to show the renumbering 
of the remaining figures. Each drawing sheet submitted after the filing date of an 
application must be labeled in the top margin as either "Replacement Sheet" or "New 
Sheet" pursuant to 37 CFR 1 .121(d). If the changes are not accepted by the examiner, 
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the applicant will be notified and informed of any required corrective action in the next 
Office action. The objection to the drawings will not be held in abeyance. 

Claim Objections 

4. Claims 6, 26, 29, 31 , and 39 are objected to because of the following 
informalities: 

In regards to claim 6, the word -wherein- should be added before the word 
"said" at line 3 of the claim. 

In regards to claim 26, the period should be deleted at line 1 of the claim. 

In regards to claim 29, the word "said" should be deleted before the word "by" at 
line 2 of the claim. 

In regards to claim 31, the period should be deleted before the word 
"multimedia" at the last line of the claim. 

In regards to claim 39, the repeated word "for" should be deleted at line 1 of the 

claim. 
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Appropriate correction is required. 

5. Applicant is advised that should claims 6 and 10 be found allowable, claims 15 
and 16, respectively, will be objected to under 37 CFR 1.75 as being a substantial 
duplicate thereof. When two claims in an application are duplicates or else are so close 
in content that they both cover the same thing, despite a slight difference in wording, it 
is proper after allowing one claim to object to the other as being a substantial duplicate 
of the allowed claim. See MPEP § 706.03(k). 

Claim Rejections - 35 USC § 101 

6. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

7. Claims 1-26 and 31-39 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. 

Claims 1-26 and 31-39 appear to be directed towards abstract ideas and do not 
appear to produce a useful, concrete and tangible result. 

The examiner will apply prior art to these claims as best understood, with the 
assumption that applicant will amend to overcome the stated 101 rejections. 
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Claim Rejections - 35 USC § 102 

8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

9. Claims 1-40 are rejected under 35 U.S.C. 102(b) as being anticipated by Liu et 
al., "Queries of Digital Content Descriptions in MPEG-7 and MPEG-21 XML 
documents," pages 1-13, XML Europe 2002, May 22, 2002 (hereinafter "Liu"). 

In regards to claim 1, Liu teaches a query system for structured multimedia content 
retrieval, said system including: 

- a query language having query constructs and formalisms for specifying 
characteristics of extensible markup language (XML) documents for retrieval 
(Liu; Section 1, Introduction, paragraph 3, lines 1-8); and 

- wherein said characteristics include spatial, temporal, and visual datatypes (Liu; 
Section 1, Introduction, paragraph 1, lines 4-5; Introduction, paragraph 3, 
lines 7-8). 

In regards to claim 2, Liu teaches a query system as recited in claim 1, wherein said 
query language includes: 



Application/Control Number: 10/609,257 Page 6 

Art Unit: 2161 

means for resolving intensional data and relationships arising from any of (Liu; 
Section 3.1, Predicate-based Query Specifications, paragraph 1, lines 1-11): 

(a) XML datatype mechanism (Liu; Section 3, Query Specification Issues in 
MPEG-7 and MPEG-21, bullet 1, lines 3-5, bullet 2, lines 4-5; Section 3.1, 
Predicate-based Query Specifications, paragraph 1, lines 1-2); 

(b) irregular XML structures (Liu; Section 3, Query Specification Issues in 
MPEG-7 and MPEG-21, bullet 2, lines 1-2; Section 3.1, Predicate-based Query 
Specifications, paragraph 1, lines 1-2); and 

(c) co-occurrence constraints (Liu; Section 3, Query Specification Issues in 
MPEG-7 and MPEG-21, bullet 3, lines 1-2; Section 3.1, Predicate-based Query 
Specifications, paragraph 1, lines 1-2). 

In regards to claim 3, Liu teaches a query system as recited in claim 2, wherein said 
means for resolving comprises a logic formalism for supporting queries on XML 
documents with any of (Liu; Section 3.1, Predicate-based Query Specifications, 
paragraph 1, lines 1-11): 

(A) intensional data and relationships (Liu; Section 3, Query Specification Issues 
in MPEG-7 and MPEG-21, bullet 1, line 1; Section 3.1, Predicate-based Query 
Specifications, paragraph 1, lines 1-2); 

(B) irregular document structures (Liu; Section 3, Query Specification Issues in 
MPEG-7 and MPEG-21, bullet 2, lines 1-2; Section 3.1, Predicate-based Query 
Specifications, paragraph 1, lines 1-2); and 
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(C) and co-occurrence constraints (Liu; Section 3, Query Specification Issues in 
MPEG-7 and MPEG-21, bullet 3, line 1; Section 3.1, Predicate-based Query 
Specifications, paragraph 1, lines 1-2). 

In regards to claim 4, Liu teaches a query system as recited in claim 1, wherein said 
query language includes means for identifying specification issues in XML query 
language for XML document retrieval (Liu; Section 3, Query Specification Issues in 
MPEG-7 and MPEG-21, paragraph 1, lines 1-3; Section 3.1, Predicate-based Query 
Specifications, paragraph 1, lines 1-2). 

In regards to claim 5, Liu teaches a query system as recited in claim 1 , wherein said 
query language includes means for identifying specification issues in XML query 
language for MPEG-7 document retrieval (Liu; Section 2.2, Spatial and Temporal 
Datatypes, paragraph 2, lines 1-2; Section 3, Query Specification Issues in MPEG- 
7 and MPEG-21, paragraph 1, lines 1-3; Section 3.1, Predicate-based Query 
Specifications, paragraph 1, lines 1-2). 

In regards to claim 6, Liu teaches a query system for structured multimedia content 
retrieval, said system including: 

a query language based on logic formalism for content retrieval (Liu; Section 1, 
Introduction, paragraph 3, lines 1-8); and 
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said logic formalism including atomic logic formulas, said atomic logic formulas being 
element predicates in a relational calculus (Liu; Section 1, Introduction, paragraph 3, 
lines 1-8). 

In regards to claim 7, Liu teaches a query system as recited in claim 6, wherein said 
query language comprises query constructs and formalisms for specifying different 
aspects of extensible markup language (XML) documents (Liu; Section 1, 
Introduction, paragraph 3, lines 1-8). 

In regards to claim 8, Liu teaches a query system as recited in claim 7, wherein said 
query constructs and formalisms are adapted for spatial, temporal and visual datatypes 
(Liu; Section 1, Introduction, paragraph 1, lines 4-5; Introduction, paragraph 3, 
lines 7-8). 

In regards to claim 9, Liu teaches a query system as recited in claim 7, wherein said 
query constructs and formalisms are adapted for spatial, temporal and visual datatypes 
in MPEG-7 documents (Liu; Section 1, Introduction, paragraph 1, lines 4-5; 
Introduction, paragraph 3, lines 7-8). 



In regards to claim 10, Liu teaches a query system as recited in claim 6, wherein: 



Application/Control Number: 10/609,257 Page 9 

Art Unit: 2161 

queries in said relational calculus are equivalent to a proof-finding process (Liu; 
Section 3.1, Predicate-based Query Specifications, paragraph 1, lines 1-11, 
paragraph 3, lines 8-9); and 

said proof-finding process comprises finding all proofs to existential closure of logical 
assertions in the form of path predicates required to be satisfied by tree document 
elements (Liu; Section 3.1, Predicate-based Query Specifications, paragraph 1, 
lines 1-11, paragraph 3, lines 8-9). 

In regards to claim 11, Liu teaches a query system as recited in claim 7, wherein 
spatial, temporal and visual datatypes and relationships are described in said logic 
formalism for content retrieval (Liu; Section 1, Introduction, paragraph 3, lines 1-8). 

Claims 12, 13, and 14 are rejected with the same rationale given for claims 2, 4, 
and 5, respectively. 

Claims 15, 16, 17, 18, 19, and 20 are rejected with the same rationale given for 
claims 6, 10, 11, 12, 13, and 14, respectively. 

In regards to claim 21, Liu teaches a query system for structured multimedia 
content retrieval, said system including: 

- a query language based on logic formalism for content retrieval (Liu; Section 1, 
Introduction, paragraph 3, lines 1-8); 
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- said logic formalism including atomic logic formulas, said atomic logic formulas 
being element predicates in a relational calculus (Liu; Section 1, Introduction, 
paragraph 3, lines 1-8); 

- means for identifying given specifications of multimedia XML documents in 
MPEG-7 XML query specifications (Liu; Section 1, Introduction, paragraph 3, 
lines 1-8; Section 3.1, Predicate-based Query Specifications, paragraph 1, 
lines 1-11); and 

- means for applying said logic formalism for processing said given specifications 
for specifying spatial and temporal relationships pertaining to said XML 
documents to support MPEG-7 XML document retrieval and modification of 
multimedia XML documents (Liu; Section 1, Introduction, paragraph 3, lines 
1-8; Section 3.1, Predicate-based Query Specifications, paragraph 3, line 1; 
Section 4, MPEG-7 and MPEG-21 Structured Queries, paragraph 1, line 3 - 
paragraph 2, line 3). 

In regards to claim 22, Liu teaches a query system in accordance with claim 21 , 
wherein said given specifications include intensional data and relationships 
specifications, document addressing specifications, and co-occurrence constraints 
specifications (Liu; Section 3, Query Specification Issues in MPEG-7 and MPEG-21, 
bullets 1, 2, and 3; Section 3.1, Predicate-based Query Specifications, paragraph 
1, lines 1-2). 
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In regards to claim 23, Liu teaches a query system in accordance with claim 21, 
wherein said given specifications include element datatypes (Liu; Section 3, Query 
Specification Issues in MPEG-7 and MPEG-21, bullet 1, lines 3-5; Section 3.1, 
Predicate-based Query Specifications, paragraph 1, lines 1-11). 

In regards to claim 24, Liu teaches a query system in accordance with claim 23, 
wherein spatial and temporal relationships are derived from said element datatypes 
(Liu; Section 2.2, Spatial and Temporal Datatypes, paragraph 1, lines 1-8). 

In regards to claim 25, Liu teaches a query system in accordance with claim 24, 
wherein spatial and temporal relationships are further included in said given 
specifications as a complex datatype for multimedia XML documents (Liu; Section 2.2, 
Spatial and Temporal Datatypes, paragraph 1, lines 1-8). 

In regards to claim 26, Liu teaches a query system in accordance with claim 23, 
wherein said datatypes.include: 

(A) Mpeg7Type, basic datatypes, reference datatypes, unique identifier datatypes, 
and time datatypes (Liu; Section 2.2, Spatial and Temporal Datatypes, paragraph 3, 
lines 1-2); 

(B) MPEG-7 visual datatypes used to specify visual properties of multimedia objects, 
including spatial, color, texture, motion, location (Liu; Section 2.2, Spatial and 
Temporal Datatypes, paragraph 5, lines 1-2); and 
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(C) MPEG-7 audio datatypes are used to specify audio content (Liu; Section 2.2, 
Spatial and Temporal Datatypes, paragraph 6, line 1). 

In regards to claim 27, Liu teaches a query system in accordance with claim 21, 
including a tool for generating a description from a video based on a scene change 
technique (Liu; Section 2.1, MPEG-7 XML Documents for Multimedia Content 
Descriptions, paragraph 4, lines 1-2), said tool including processing means for: 

(a) breaking down the video temporally into scenes or shots using scene change 
detection algorithms that can detect both abrupt as well as gradual changes (Liu; 
Section 2.1, MPEG-7 XML Documents for Multimedia Content Descriptions, 
paragraph 4, lines 2-3); 

(b) outlining user-identified objects of interest within said scenes (Liu; Section 2.1, 
MPEG-7 XML Documents for Multimedia Content Descriptions, paragraph 4, lines 
3-4); 

(c) tracking said user-identified objects (Liu; Section 2.1, MPEG-7 XML 
Documents for Multimedia Content Descriptions, paragraph 4, line 4); 

(d) creating a node point where a significant motion change wherein a linear mode is 
inadequate (Liu; Section 2.1, MPEG-7 XML Documents for Multimedia Content 
Descriptions, paragraph 4, lines 4-5); 

(e) providing the specification of said user-identified objects as any of temporal, 
audio, and visual datatypes (Liu; Section 2.1, MPEG-7 XML Documents for 
Multimedia Content Descriptions, Figure 2, paragraph 3); and 
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(f) providing a description of said user-defined objects as any of spatial, temporal 
and visual datatypes (Liu; Section 2.1, MPEG-7 XML Documents for Multimedia 
Content Descriptions, Figure 2, paragraph 3; Section 2.2, Spatial and Temporal 
Datatypes, paragraph 1, lines 1-2). 

In regards to claim 28, Liu teaches a query system in accordance with claim 27, 
wherein processing means provides said tracking said user-identified objects in a semi- 
automatic manner (Liu; Section 2.1, MPEG-7 XML Documents for Multimedia 
Content Descriptions, paragraph 4, line 4). 

In regards to claim 29, Liu teaches a query system in accordance with claim 27, 
wherein processing means provides said description of said user-defined objects said 
by the use of abstract datatype techniques (ADT) (Liu; Section 2.2, Spatial and 
Temporal Datatypes, paragraph 1, lines 1-2). 

In regards to claim 30, Liu teaches a query system in accordance with claim 27, 
wherein processing means provides said respective datatypes as composite datatypes 
constructed from more primitive ones (Liu; Section 2.2, Spatial and Temporal 
Datatypes, paragraph 1, lines 2-3). 

In regards to claim 31, Liu teaches a query system for multimedia content retrieval, 
said system including: 
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- a query language based on logic formalism for content retrieval, said logic 
formalism being hereinafter referred to as Path Predicate Calculus and being 
utilized for logic-based queries and manipulations (Liu; Section 1, Introduction, 
paragraph 3, lines 1-8; Section 3.1, Predicate-based Query Specifications, 
paragraph 1, lines 1-11, paragraph 3, line 1); 

- said Path Predicate Calculus including atomic logic formulas, said atomic logic 
formulas being element predicates in a relational calculus and comprising 
element predicates and path predicates, for asserting logical truth statements 
about document elements in a document tree (Liu; Section 1, Introduction, 
paragraph 3, lines 1-8; Section 3.1, Predicate-based Query Specifications, 
paragraph 1, lines 1-11); 

- means for identifying given specifications of multimedia XML documents in 
MPEG-7 XML query specifications (Liu; Section 1, Introduction, paragraph 3, 
lines 1-8; Section 3.1, Predicate-based Query Specifications, paragraph 1, 
lines 1-11); and 

- means for applying said logic formalism for processing said given specifications 
for specifying spatial and temporal relationships pertaining to said XML 
documents to support MPEG-7 XML document retrieval and modification, 
multimedia XML documents (Liu; Section 1, Introduction, paragraph 3, lines 
1-8; Section 3.1, Predicate-based Query Specifications, paragraph 3, line 1; 
Section 4, MPEG-7 and MPEG-21 Structured Queries, paragraph 1, line 3 - 
paragraph 2, line 3). 
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Claims 32, 33, 34, and 35 are rejected with the same rationale given for claims 10, 
11, 12, and 13, respectively. 

In regards to claim 36, Liu teaches a query system for structured multimedia 
content retrieval, said system including: 

- a query language based on logic formalism for content retrieval, said language 
including query constructs and formalisms for specifying different aspects of XML 
documents (Liu; Section 1, Introduction, paragraph 3, lines 1-8); and 

- wherein said constructs and formalisms are particularly adapted for spatial, 
temporal and visual datatypes (Liu; Section 1, Introduction, paragraph 1, lines 
4-5; Introduction, paragraph 3, lines 7-8). 

In regards to claim 37, Liu teaches a query system as recited in claim 36, wherein 
said query language (Liu; Section 3.1, Predicate-based Query Specifications, 
paragraph 1, lines 1-11) identifies intensional data and relationships due to XML 
datatype mechanisms (Liu; Section 3, Query Specification Issues in MPEG-7 and 
MPEG-21, bullet 1, lines 3-5, bullet 2, lines 4-5; Section 3.1, Predicate-based Query 
Specifications, paragraph 1, lines 1-2), irregular XML structures (Liu; Section 3, 
Query Specification Issues in MPEG-7 and MPEG-21, bullet 2, lines 1-2; Section 
3.1, Predicate-based Query Specifications, paragraph 1, lines 1-2), and co- 
occurrence constraints (Liu; Section 3, Query Specification Issues in MPEG-7 and 
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MPEG-21, bullet 3, lines 1-2; Section 3.1, Predicate-based Query Specifications, 
paragraph 1, lines 1-2) for document retrieval. 

In regards to claim 38, Liu teaches a query system as recited in claim 37, wherein 
said query language is specially adapted for MPEG-7 documents (Liu; Section 1, 
Introduction, paragraph 3, lines 1-8; Section 3.1, Predicate-based Query 
Specifications, paragraph 2, line 1). 

Claims 39 and 40 are rejected with the same rationale given for claims 21 and 27, 
respectively. 

Conclusion 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kavita Padmanabhan whose telephone number is 571- 
272-8352. The examiner can normally be reached on Monday-Friday, 9:00am-5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Safet Metjahic can be reached on 571-272-4023. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 



Application/Control Number: 10/609,257 Page 17 

Art Unit: 2161 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Kavita Padmanabhan December 30, 2005 

Assistant Examiner 

AU2161 

if- 



SAFET METJAHIC 
SUPERVISORY PATENT EXAMINER 
ETHNOLOGY CENTER 2100 



